Sanchy - welcome and look forward to your contributions. As many others have said, you are not alone! Your story sounds similar to mine and many others here.
It took me much longer to step down and you've probably saved yourself some sleepless nights, although with a new baby on the way, you'll be getting plenty of those!!
Just don't make any drastic decisions and if you decide to fade, DON'T mention doubts anymore. The stress of taking care of a young family should be enough to get them off your back.
I feel one of the biggest advantages of this site is that it allows people to vent in a way they really can't do with people they know!